Coefficient coding harmonization in hevc

ABSTRACT

Coefficient coding for transform units (TUs) during high efficiency video coding (HEVC), and similar standards, toward simplifying design while enhancing efficiency. Elements of the invention include coefficient coding for TUs with up-right diagonal scans being modified, and selectively applying multi-level significance map coding.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority from U.S. patent applicationSer. No. 15/919,703, filed Mar. 13, 2018, which is a continuation ofU.S. patent application Ser. No. 15/686,372, filed Aug. 25, 2017, whichis a continuation of U.S. patent application Ser. No. 14/792,300, filedJul. 6, 2015 (now U.S. Pat. No. 9,826,236), which is a continuation ofSer. No. 13/631,464, filed Sep. 28, 2012 (now U.S. Pat. No. 9,106,918),which claims priority from U.S. provisional patent application Ser. No.61/589,151 filed Jan. 20, 2012, each of which are incorporated herein byreference in its entirety.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not Applicable

INCORPORATION-BY-REFERENCE OF MATERIAL SUBMITTED ON A COMPACT DISC

Not Applicable

NOTICE OF MATERIAL SUBJECT TO COPYRIGHT PROTECTION

A portion of the material in this patent document is subject tocopyright protection under the copyright laws of the United States andof other countries. The owner of the copyright rights has no objectionto the facsimile reproduction by anyone of the patent document or thepatent disclosure, as it appears in the United States Patent andTrademark Office publicly available file or records, but otherwisereserves all copyright rights whatsoever. The copyright owner does nothereby waive any of its rights to have this patent document maintainedin secrecy, including without limitation its rights pursuant to 37C.F.R. § 1.14.

BACKGROUND OF THE INVENTION 1. Field of the Invention

This invention pertains generally to video coding, and more particularlyto transform unit (TU) encoding and decoding within high efficiencyvideo coding standards.

2. Description of Related Art

In high efficiency video coding (HEVC), a coding unit (CU) may havevariable sizes depending on video content toward achieving a desiredcoding efficiency. CUs typically include a luminance component, Y, andtwo chroma components, U and V. The size of U and V components relate tothe number of samples, and can be the same or different from that of theY component, as depends upon the video sampling format. These codingunits may be split into smaller blocks for prediction or transform. Inparticular, each coding unit may be further partitioned into predictionunits (PUs) and transform units (TUs). Prediction units (PU) can bethought of similarly to partitions described in other video codingstandards, such as the H.264 standard. Transform units (TU) generallyrefer to a block of residual data to which a transform is applied whengenerating transform coefficients.

Transform unit (TU) coding within high efficiency video coding (HEVC),requires complex coding steps with significant processing overhead andgenerally comprise several steps including: mode dependent coefficientscan (MDCS), last non-zero coefficient coding, significance map codingand non-zero coefficient level coding. These components vary atdifferent transform unit (TU) sizes.

Accordingly, a need exists for simplifying the design of HEVC coding.The present invention fulfills that need as well as others, towardimproving HEVC coding operations.

BRIEF SUMMARY OF THE INVENTION

Coefficient coding for transform units (TUs) is described which enhancesand harmonizes overall operation across 4×4, 8×8, 16×16 and 32×32 TUs.In a first portion, coefficient coding for TUs with up-right diagonalscans is modified, and a second portion applies a multi-levelsignificance map coding. Both of these inventive elements apply to TUswith a size of 4×4 or 8×8.

Further aspects of the invention will be brought out in the followingportions of the specification, wherein the detailed description is forthe purpose of fully disclosing preferred embodiments of the inventionwithout placing limitations thereon.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)

The invention will be more fully understood by reference to thefollowing drawings which are for illustrative purposes only:

FIG. 1 is a schematic of a video encoder according to an embodiment ofthe present invention.

FIG. 2 is a schematic of a video decoder according to an embodiment ofthe present invention.

FIG. 3 is a flow diagram of generalized transform unit (TU) encodingsteps utilized according to an embodiment of the present invention.

FIG. 4 is a flow diagram of generalized TU decoding steps utilizedaccording to an embodiment of the present invention.

FIG. 5 is a flow diagram of a conventional mode dependent coefficientscan (MDCS).

FIG. 6 is a flow diagram of a mode dependent coefficient scan (MDCS)according to an embodiment of the present invention.

FIGS. 7A, 7B, 7C, and 7D are transform unit scanning patterns utilizedconventionally, and according to embodiments of the present invention.

FIG. 8 is a flow diagram of conventional significance map encoding.

FIG. 9 is a flow diagram of conventional significance map decoding.

FIG. 10 is a flow diagram of significance map encoding according to anembodiment of the present invention.

FIG. 11 is a flow diagram of significance map decoding according to anembodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

During high efficiency video coding (HEVC), in particular HEVC testmodel HM5.0, coefficient coding consists of several steps which vary atdifferent transform unit (TU) sizes.

To enhance the operation of HEVC coding, a more unified solution tocoefficient coding is taught herein that harmonizes coefficient coding,such as for the up-right diagonal scan, so that all TUs with up-rightdiagonal scan will have the same coefficient coding.

Table 1 and Table 2 compare elements from existing coefficient coding inHEVC test model HM5.0 (Table 1) with changes according to the inventionseen in Table 2. The column marked scan is the transform coefficientscanning order, and the Multi-level Sig Map represents how Multi-levelsignificance map coding first encodes a CG flag. It will be noted thatin moving from Table 1 to Table 2 that scanning is enhanced in certaininstances with 4×4 and 8×8 TU sizes with sub-block up-right diagonalscanning (sub-D), while additional application of multi-levelsignificance mapping is applied.

FIG. 1 illustrates an example embodiment of a coding apparatuscomprising an encoder 10 according to the invention for performingreplacement of up-right diagonal scan (RDS), and for applyingmulti-level significance map coding (MLSMC). The invention isimplemented within the entropy encoding block 34, shown containinggeneralized RDS and MLSMC, but otherwise can rely on conventional videocoding which maximizes compatibility with coding systems.

The encoder 10 is shown with encoding elements 12 executed by one ormore processors 44. In the example, video frame input 14 is shown alongwith reference frames 16 and frame output 18. Inter-prediction 20 isdepicted with motion estimation (ME) 22 and motion compensation (MC) 24.Intra prediction 26 is shown and switching is depicted between interprediction and intra prediction. A sum junction 28 is shown with outputto a forward transform 30 which is performed based on the predictions togenerate transform coefficients of residual data. Quantization of thetransform coefficients is performed at quantization stage 32, which isfollowed by entropy encoding 34. Inverse quantization 36 and inversetransform 38 operations are shown coupled to a summing junction 40followed by a filter 42, such as a deblocking and/or loop filter and/orsample adaptive offset.

It should be appreciated that the encoder is shown implemented with aprocessing means 44, such as comprising at least one processing device(e.g., CPU) 46 and at least one memory 48 for executing programmingassociated with the encoding. In addition, it will be appreciated thatelements of the present invention can be implemented as programmingstored on a media, which can be accessed for execution by a CPU for theencoder 10 and/or decoder 50.

FIG. 2 illustrates an example embodiment 50 of a decoder, shown withprocess blocks 52 and an associated processing means 76. It will benoted that the decoder is substantially a subset of the elementscontained in encoder 10 of FIG. 1, operating on reference frames 54 andoutputting video 74. The decoder blocks receive an encoded video signal56 which is processed through entropy decoder 58 which performs decodingof the one dimensional TUs based on the mode dependent scan and decodingof the last non-zero transform position as determined by the encoder.The TUs are processed: (1) during mode dependent coefficient scanning(MDCS) with TUs that are 4×4 or 8×8 horizontal or vertical subject tohorizontal or vertical scanning and the remaining TUs, including theup-right diagonal 4×4 and 8×8 TUs subject to 4×4 sub-block up-rightdiagonal scanning; or (2) using multi-level significance maps for bothlarge TUs, and 4×4 and 8×8 TUs with up-right diagonal scans. Duringusing the multi-level significance maps the programming of the decoderdecodes a flag from the encoder which determines if a coefficient groupis all zero or not, and selects an individual significance map if thecoefficient group has any non-zero coefficients.

Following entropy decoding is inverse quantization 60, inverse transform62, and summing 64 between the inverse transform 62 output and theselection between inter prediction 66 shown with motion compensation 68,and a separate intra prediction block 70. Output from summing junction64 is received by filter 72, which can be configured as a loop filter, adeblocking filter, sample adaptive offset or any combination thereof. Itshould be appreciated that the decoder can be implemented with aprocessing means 76 which comprises at least one processing device 78and at least one memory 80 for executing programming associated with thedecoding. In addition, it will be noted that elements of the presentinvention can be implemented as programming stored on a media, whereinsaid media can be accessed for execution by processing device (CPU) 78.

It will be recognized that elements of the present invention 10 and 50are implemented for execution by a processing means 44 and 76, such asin response to programming resident in memory 48 and 80 which isexecutable on computer processor (CPU) 46 and 78. In addition, it willbe appreciated that elements of the present invention can be implementedas programming stored on a media, wherein said media can be accessed forexecution by CPU 46 and 78.

It should be appreciated that the programming is executable from thememory which is a tangible (physical) computer readable media that isnon-transitory in that it does not merely constitute a transitorypropagating signal, but is actually capable of retaining programming,such as within any desired form and number of static or dynamic memorydevices. These memory devices need not be implemented to maintain dataunder all conditions (e.g., power fail) to be considered herein asnon-transitory media.

FIG. 3 illustrates general TU coding steps in an encoder which arefollowed by both conventional TU coding and TU coding according to thepresent invention. These general steps comprise converting twodimensional (2D) TU into a one dimensional (1D) TU based on a modedependent coefficient scan (MDCS) 90. The last non-zero coefficientposition is identified and encoded 92. A significance map coding 94encodes whether a coefficient is zero or non-zero. Then the values ofnon-zero coefficients are encoded 96 to complete the TU coding.

FIG. 4 illustrates general TU coding steps in a decoder which arefollowed by both conventional TU coding and TU coding according to thepresent invention. These general steps comprise converting twodimensional (2D) TU into a one dimensional (1 D) TU based on a modedependent coefficient scan (MDCS) 98. The last non-zero coefficientposition is decoded 100. A significance map coding 102 decodes whether acoefficient is zero or non-zero. Then the values of non-zerocoefficients are decoded 104 to complete the TU coding in the decoder.

FIG. 5 depicts a conventional method of performing mode dependentcoefficient scan (MDCS). TU information (e.g., size, prediction mode) isreceived 110 with and large TUs are detected in step 112, with TUs thatare not 4×4 or 8×8 being processed with 4×4 sub-block up-right diagonalscanning 114. The 4×4 and 8×8 TUs are checked at step 116, and thosethat are horizontal or vertical are processed at step 118 usinghorizontal or vertical scanning. For the 4×4 and 8×8 TUs which are nothorizontal or vertical, processing moves from block 116, to block 120where a check is made to detect 4×4 TUs. The 4×4 TUs are then processedby a 4×4 up-right diagonal scan 122, with 8×8 TUs processed by 8×8up-right diagonal scanning 124.

FIG. 6 illustrates an example embodiment of mode dependent coefficientscanning (MDCS) according to the invention. TU information is received130 with large TUs detected in step 132, and TUs that are not 4×4 or 8×8being processed with 4×4 sub-block up-right diagonal scanning 134. The4×4 and 8×8 TUs are checked at step 136, and those that are horizontalor vertical are processed at step 138 using horizontal or verticalscanning. For the remaining 4×4 and 8×8 non-horizontal, non-verticalTUs, 4×4 sub-block up-right diagonal scanning is also performed as perblock 134.

FIGS. 7A, 7B, 7C, and. 7D illustrate scanning patterns according to theinvention. In FIG. 7A a large TU is seen having 16×16 coefficients,which is subject to sub-block partitioning and up-right diagonalscanning. The figure shows that coefficients are scanned within each 4×4sub-block, and then scanning moves to the next 4×4 sub-block (i.e., CG).For the sake of simplicity (and space) the scanning pattern within eachof the 4×4 sub-blocks is not shown in FIG. 7A. In FIG. 7B an up-rightdiagonal scan is shown on a 4×4 up-right diagonal TU. It will be notedthat 4×4 up-right diagonal scanning is performed on these TUs bothbefore and after the inventive changes seen in FIG. 6. In FIG. 7C isseen a conventional 8×8 up-right diagonal scan, which is replacedaccording to the invention with 4×4 up-right diagonal sub-block scanningas seen in FIG. 7D. In FIGS. 7A, 7B, 7C, and 7D, MDCS starts from thetop-left corner of a TU and traverses through to the bottom-rightcorner. In the encoding processes 94 and 96 of FIG. 3 and the decodingprocesses 102 and 104 of FIG. 4, the processing order is the reverse ofMDCS shown in FIGS. 7A, 7B, 7C, and 7D.

FIG. 8 depicts conventional significance map processing in an encoder.TU information 150 is received and for 4×4 and 8×8 TUs, as determined instep 152, a single level significance map encoding 154 is performed.Otherwise, for TUs which are not 4×4 or 8×8, then multi-levelsignificance map encoding 156 is performed. The multi-level significancemap encoding is shown comprising checking 158 if there is any morecoefficient groups (CG) starting from the last nonzero CG. If no moreCG, then multi-level significance map encoding is completed, andexecution is seen jumping past step 166. If there are more CG, then acheck is made to see if we are between the first and last CGs at step160. It will be noted that: (1) flagging need not be sent for the allzero CG after the last nonzero CG (the CG containing the last nonzerocoefficient), as it can be presumed these will be all zeros and CG flagis set to zero; (2) no flagging is needed the last nonzero CG, as it canbe deduced that this has non zero coefficients and CG flag is set toone, and finally (3) flagging is not needed for the first CG, as inalmost all cases this CG has nonzero coefficients and CG flag is set toone. Thus, if between the first and last CG as determined in step 160,then (yes) flagging 162 is performed with CG flag encoded. If notbetween the first and last CGs, then CG flag is set to one 161 and theflagging step is bypassed and execution proceeds to the CG flag check164. In step 164 it is checked whether the CG flag is equal to one ornot, with a return to step 158 if the CG is equal to zero. It will benoted that each 4×4 sub-block comprises a coefficient group (CG). If theCG flag is equal to one, as determined at step 164, then an individualsignificance map encoding is performed 166.

FIG. 9 depicts conventional significance map processing in a decoder. TUinformation 170 is received and for 4×4 and 8×8 TUs, as determined instep 172, a single level significance map decoding 174 is performed.Otherwise, for TUs which are not 4×4 or 8×8, then multi-levelsignificance map decoding 176 is performed. The multi-level significancemap decoding is shown comprising checking 178 if there is any morecoefficient groups (CG) starting from the last nonzero CG. If no moreCG, then multi-level significance map decoding is completed, andexecution is seen jumping past step 186. If there are more CG, then acheck is made to see if we are between the first and last CGs at step180. It will be noted that: (1) flagging need not be sent for the allzero CG after the last nonzero CG (the CG containing the last nonzerocoefficient), as it can be presumed these will be all zeros and CG flagis set to zero; (2) no flagging is needed the last nonzero CG, as it canbe deduced that this has non zero coefficients and CG flag is set toone, and finally (3) flagging is not needed for the first CG, as inalmost all cases this CG has nonzero coefficients and CG flag is set toone. Thus, if between the first and last CG as determined in step 180,then (yes) flagging 182 is performed. If not between the first and lastCGs, then the CG flag is set to one 181 and the flagging step isbypassed and execution proceeds to the CG flag check 184. In step 184 itis checked whether the CG flag is equal to one or not, with a return tostep 178 if the CG is equal to zero. It will be noted that each 4×4sub-block comprises a coefficient group (CG). If the CG flag is equal toone, as determined at step 184, then an individual significance mapdecoding is performed 186.

FIG. 10 illustrates significance map processing in an encoder accordingto an element of the present invention. TU information 190 is receivedand if it is a 4×4 or 8×8 TU with horizontal or vertical scan, asdetermined in step 192, then a single level significance map encoding194 is performed. Otherwise, for large TUs, and 4×4 and 8×8 up-rightdiagonal scan TUs, multi-level significance map encoding 196 isperformed. The multi-level significant map encoding is shown comprisingchecking 198 if there is any more coefficient groups (CG). If no moreCG, then multi-level significance map encoding is completed, andexecution is seen jumping past step 206. If there are more CG, then acheck is made to see if we are between the first and last CGs at step200. It will be noted that: (1) flagging need not be sent for the allzero CG after the last nonzero CG (the CG containing the last nonzerocoefficient), as it can be presumed these will be all zeros and CG flagis set to one; (2) no flagging is needed for the last nonzero CG, as itcan be deduced that this has non zero coefficients, and finally (3)flagging is not needed for the first CG, as in almost all cases this CGhas nonzero coefficients and CG flag is set to one. Thus, if between thefirst and last CG as determined in step 200, then (yes) flagging isperformed 202. If not between the first and last CGs, then the CG flagis set to one 201 and the flagging step is bypassed with executionadvancing to the CG flag check 204. In step 204 it is checked whether CGflag is equal to one or not, with a return to step 198 if the CG isequal to zero. It will be noted that each 4×4 sub-block comprises acoefficient group (CG). If the CG is equal to one, as determined at step204, then an individual significance map encoding is performed 206.

FIG. 11 illustrates significance map processing in a decoder accordingto an element of the present invention. TU information 210 is receivedand if it is a 4×4 or 8×8 TU with horizontal or vertical scan, asdetermined in step 212, then a single level significance map decoding214 is performed. Otherwise, for large TUs, and 4×4 and 8×8 up-rightdiagonal scan TUs, multi-level significance map decoding 216 isperformed. The multi-level significant map decoding is shown comprisingchecking 218 if there is any more coefficient groups (CG). If no moreCG, then multi-level significance map decoding is completed, andexecution is seen jumping past step 226. If there are more CG, then acheck is made to see if we are between the first and last CGs at step220. It will be noted that: (1) flagging need not be sent for the allzero CG after the last nonzero CG (the CG containing the last nonzerocoefficient), as it can be presumed these will be all zeros and CG flagis set to one; (2) no flagging is needed for the last nonzero CG, as itcan be deduced that this has non zero coefficients, and finally (3)flagging is not needed for the first CG, as in almost all cases this CGhas nonzero coefficients and CG flag is set to one. Thus, if between thefirst and last CG as determined in step 220, then (yes) flagging isperformed 222 with CG flag encoded. If not between the first and lastCGs, then the CG flag is set to one 221 with the flagging step bypassedand execution proceeding to the CG flag check 224. In step 224 it ischecked whether CG flag is equal to one or not, with a return to step218 if the CG is equal to zero. It will be noted that each 4×4 sub-blockcomprises a coefficient group (CG). If the CG is equal to one, asdetermined at step 224, then an individual significance map decoding isperformed 226.

The following summarizes moving from FIG. 8 (encoder) and FIG. 9(decoder) to the inventive teachings of FIG. 10 (encoder) and FIG. 11(decoder). In the existing significance mapping, all 4×4 and 8×8 TUs aresubject to single level significance mapping, while only larger TUs(16×16 and 32×32) are processed by multi-level significance mapping.However, as seen in FIGS. 10 and 11, 4×4 and 8×8 TUs with horizontal orvertical scans are processed with a single level significance map, while4×4 and 8×8 TUs with up-right diagonal scans along with the large TUs(16×16 and 32×32) are processed using the multi-level significancemapping.

These solutions are summarized in the characteristics of Table 2. It canbe seen in the table that certain 4×4 and 8×8 TUs are processeddifferently than before with sub-block up-right diagonal (Sub-D)scanning. The inventive technique was implemented into HEVC HM5.0, witha simulation conducted under common test conditions.

Embodiments of the present invention may be described with reference toflowchart illustrations of methods and systems according to embodimentsof the invention, and/or algorithms, formulae, or other computationaldepictions, which may also be implemented as computer program products.In this regard, each block or step of a flowchart, and combinations ofblocks (and/or steps) in a flowchart, algorithm, formula, orcomputational depiction can be implemented by various means, such ashardware, firmware, and/or software including one or more computerprogram instructions embodied in computer-readable program code logic.As will be appreciated, any such computer program instructions may beloaded onto a computer, including without limitation a general purposecomputer or special purpose computer, or other programmable processingapparatus to produce a machine, such that the computer programinstructions which execute on the computer or other programmableprocessing apparatus create means for implementing the functionsspecified in the block(s) of the flowchart(s).

Accordingly, blocks of the flowcharts, algorithms, formulae, orcomputational depictions support combinations of means for performingthe specified functions, combinations of steps for performing thespecified functions, and computer program instructions, such as embodiedin computer-readable program code logic means, for performing thespecified functions. It will also be understood that each block of theflowchart illustrations, algorithms, formulae, or computationaldepictions and combinations thereof described herein, can be implementedby special purpose hardware-based computer systems which perform thespecified functions or steps, or combinations of special purposehardware and computer-readable program code logic means.

Furthermore, these computer program instructions, such as embodied incomputer-readable program code logic, may also be stored in acomputer-readable memory that can direct a computer or otherprogrammable processing apparatus to function in a particular manner,such that the instructions stored in the computer-readable memoryproduce an article of manufacture including instruction means whichimplement the function specified in the block(s) of the flowchart(s).The computer program instructions may also be loaded onto a computer orother programmable processing apparatus to cause a series of operationalsteps to be performed on the computer or other programmable processingapparatus to produce a computer-implemented process such that theinstructions which execute on the computer or other programmableprocessing apparatus provide steps for implementing the functionsspecified in the block(s) of the flowchart(s), algorithm(s), formula(e),or computational depiction(s).

From the discussion above it will be appreciated that the invention canbe embodied in various ways, including the following:

1. A system of encoding and decoding of video signals, comprising: (a) avideo encoder having a computer processor configured for encoding ofvideo frames; and (b) programming executable on said computer processorfor: (i) performing intra-prediction on said video frames based onneighboring reconstructed values, and/or inter-prediction on said videoframes based on motion-estimation and motion-compensation; (ii)executing a transform based on said inter-prediction followed byquantization of transform coefficients, and/or executing inversequantization followed by an inverse transform which is summed with saidinter-prediction, on said video frames; and (iii) performing entropyencoding on said video frames comprising: (A) converting two dimensionaltransform units (TUs) into one dimensional TUs based on a mode dependentcoefficient scan (MDCS); (B) identifying and encoding a last non-zerotransform coefficient position; (C) generating a significance mapencoding whether a coefficient is zero or non-zero; (D) wherein said TUsare processed either: (1) during mode dependent coefficient scanning(MDCS) with TUs that are 4×4 or 8×8 horizontal or vertical subject tohorizontal or vertical scanning and the remaining TUs, including theup-right diagonal 4×4 and 8×8 TUs subject to 4×4 sub-block up-rightdiagonal scanning; or (2) during generating significance map both largeTUs, and 4×4 and 8×8 TUs with 4×4 sub-block up-right diagonal scans, areprocessed with multi-level significance map encoding; (E) encodingremaining non-zero transform coefficients; (c) a decoder having acomputer processor configured for decoding video frames; and (d)programming executable on said computer processor for performing entropydecoding upon receiving encoded video frames comprising replacement ofup-right diagonal scan (RDS), and/or applying multi-level significancemap coding during decoding of transform units (TUs).

2. The system of any of the previous embodiments, wherein said 4×4sub-block is a coefficient group (CG).

3. The system of any of the previous embodiments, wherein during saidmulti-level significance mapping said programming is configured forexecution on said computer for encoding a flag indicating if acoefficient group is all zero or not, and generating an individualsignificance map if the coefficient group has any non-zero coefficients.

4. The system of any of the previous embodiments, wherein said encodercoding units (CUs) have variable sizes depending on video content towardachieving a desired coding efficiency, with said CUs split into smallerprediction units (PUs) and transform units (TUs).

5. The system of any of the previous embodiments, wherein said transformunits (TU) refer to a block of residual data to which said transform isapplied that generates transform coefficients.

6. The system of any of the previous embodiments, wherein said videocoding system operates with High Efficiency Video Coding (HEVC) coders.

7. The system of any of the previous embodiments, wherein saidprogramming is configured to utilize the same coefficient coding for allTUs with up-right diagonal scans.

8. An apparatus for entropy encoding within a video coding device,comprising: (a) an encoder having a computer processor configured forencoding video frames; (b) programming executable on said computerprocessor for: (i) performing inter-prediction on said video framesbased on motion-estimation and motion-compensation; (ii) executing atransform based on said inter-prediction followed by quantization oftransform coefficients from said transform; and (iii) performing entropyencoding on said video frames comprising: (A) converting two dimensionaltransform units (TUs) into one dimensional TUs based on a mode dependentcoefficient scan (MDCS); (B) identifying and encoding a last non-zerotransform coefficient position; (C) generating a significance mapencoding whether a coefficient is zero or non-zero; (D) wherein said TUsare processed either: (1) during mode dependent coefficient scanning(MDCS) with TUs that are 4×4 or 8×8 horizontal or vertical subject tohorizontal or vertical scanning and the remaining TUs, including theup-right diagonal 4×4 and 8×8 TUs subject to 4×4 sub-block up-rightdiagonal scanning; or (2) during generating significance mapping bothlarge TUs, and 4×4 and 8×8 TUs with 4×4 sub-block up-right diagonalscans, are processed with multi-level significance map encoding; and (E)encoding remaining non-zero transform coefficients.

9. The apparatus of any of the previous embodiments, wherein said 4×4sub-block is a coefficient group (CG).

10. The apparatus of any of the previous embodiments, wherein duringsaid multi-level significance mapping said programming is configured forexecution on said computer for encoding a flag indicating if acoefficient group is all zero or not, and generating an individualsignificance map if the coefficient group has any non-zero coefficients.

11. The apparatus of any of the previous embodiments, wherein codingunits (CUs) of said encoder have variable sizes depending on videocontent toward achieving a desired coding efficiency, with said CUssplit into smaller prediction units (PUs) and transform units (TUs).

12. The apparatus of any of the previous embodiments, wherein saidtransform units (TUs) refer to a block of residual data to which saidtransform is applied that generates transform coefficients.

13. The apparatus of any of the previous embodiments, wherein said videocoding apparatus operates with High Efficiency Video Coding (HEVC)coders.

14. The apparatus of any of the previous embodiments, wherein saidprogramming is configured to use the same coefficient coding for all TUswith up-right diagonal scans.

15. An apparatus for entropy decoding within a video decoding device,comprising: (a) a decoder having a computer processor configured fordecoding video frames; (b) programming executable on said computerprocessor for performing entropy decoding upon receiving encoded videoframes comprising: (i) converting two dimensional transform units (TUs)into one dimensional TUs based on a mode dependent coefficient scan(MDCS); (ii) decoding a last non-zero transform coefficient position;(iii) decoding a significance map from the encoder as to whether acoefficient is zero or non-zero; (iv) wherein said TUs are processedeither: (1) during mode dependent coefficient scanning (MDCS) with TUsthat are 4×4 or 8×8 horizontal or vertical subject to horizontal orvertical scanning and the remaining TUs, including the up-right diagonal4×4 and 8×8 TUs subject to 4×4 sub-block up-right diagonal scanning; or(2) using multi-level significance map decoding for both large TUs, and4×4 and 8×8 TUs with 4×4 sub-block up-right diagonal scans; and (v)decoding remaining non-zero transform coefficients.

16. The apparatus of any of the previous embodiments, wherein said 4×4sub-block is a coefficient group (CG).

17. The apparatus of any of the previous embodiments, wherein duringutilizing said multi-level significance maps said programming isconfigured for decoding a flag from an encoder which determines if acoefficient group is all zero or not, and selecting an individualsignificance map if the coefficient group has any non-zero coefficients.

18. The apparatus of any of the previous embodiments, wherein codingunits (CUs) of said decoder have variable sizes depending on videocontent toward achieving a desired coding efficiency, with said CUssplit into smaller prediction units (PUs) and transform units (TUs).

19. The apparatus of any of the previous embodiments, wherein saidtransform units (TUs) refer to a block of residual data to which aninverse-transform is applied.

20. The apparatus of any of the previous embodiments, wherein said videodecoding apparatus operates with High Efficiency Video Coding (HEVC)coders.

Although the description above contains many details, these should notbe construed as limiting the scope of the invention but as merelyproviding illustrations of some of the presently preferred embodimentsof this invention. Therefore, it will be appreciated that the scope ofthe present invention fully encompasses other embodiments which maybecome obvious to those skilled in the art, and that the scope of thepresent invention is accordingly to be limited by nothing other than theappended claims, in which reference to an element in the singular is notintended to mean “one and only one” unless explicitly so stated, butrather “one or more.” All structural and functional equivalents to theelements of the above-described preferred embodiment that are known tothose of ordinary skill in the art are expressly incorporated herein byreference and are intended to be encompassed by the present claims.Moreover, it is not necessary for a device or method to address each andevery problem sought to be solved by the present invention, for it to beencompassed by the present claims. Furthermore, no element, component,or method step in the present disclosure is intended to be dedicated tothe public regardless of whether the element, component, or method stepis explicitly recited in the claims. No claim element herein is to beconstrued under the provisions of 35 U.S.C. 112, sixth paragraph, unlessthe element is expressly recited using the phrase “means for.”

TABLE 1 Existing Coefficient coding in HEVC Test Model HM5.0 Multi-levelTU sizes Scan Sig Map 4 × 4 H/V/D N/A 8 × 8 H/V/D N/A 16 × 16/16 × 4/4 ×16 Sub-D applied 32 × 32/32 × 8/8 × 32 Scan: H = horizontal scan; V =vertical scan; D = up-right diagonal scan varying based on TU size;sub-D = 4 × 4 sub-block up-right diagonal scan.

TABLE 2 Inventive Coefficient coding for HEVC Multi-level TU sizes ScanSig Map 4 × 4 H/V N/A 8 × 8 H/V N/A 4 × 4 Sub-D applied 8 × 8 Sub-Dapplied 16 × 16/16 × 4/4 × 16 Sub-D applied 32 × 32/32 × 8/8 × 32 Scan:H = horizontal scan; V = vertical scan; D = up-right diagonal scanvarying based on TU size; sub-D = 4 × 4 sub-block up-right diagonalscan.

What is claimed is:
 1. A decoding device comprising: circuitryconfigured to: apply, as a condition that diagonal scan is applied to atransform block of variable block size, diagonal scan to all transformblocks of a set of variable block sizes, wherein 4×4 sub-blocks arescanned using a diagonal scan and the diagonal scan is applied insideeach 4×4 sub-block; and apply multi-level significance map decoding tothe transform block.
 2. The decoding device according to claim 1,wherein the circuitry is further configured to perform unifiedmulti-level significance map decoding on the transform block.
 3. Thedecoding device according to claim 2, wherein the set of variable blocksizes comprises 4×4, 8×8, 16×16, and 32×32 blocks.
 4. The decodingdevice according to claim 3, wherein the transform block is based onrecursively splitting a coding block into transform blocks by blockpartitioning.
 5. A decoding method comprising: applying, as a conditionthat diagonal scan is applied to a transform block of variable blocksize, diagonal scan to all transform blocks of a set of variable blocksizes, wherein 4×4 sub-blocks are scanned using a diagonal scan and thediagonal scan is applied inside each 4×4 sub-block; and applyingmulti-level significance map decoding to the transform block.
 6. Themethod according to claim 5, further comprising: performing unifiedmulti-level significance map decoding on the transform block.
 7. Themethod according to claim 6, wherein the set of variable block sizescomprises 4×4, 8×8, 16×16, and 32×32 blocks.
 8. The method according toclaim 7, further comprising: forming the transform block based onrecursively splitting a coding block into transform block by blockpartitioning.
 9. A non-transitory computer-readable medium having storedthereon, computer-executable instructions, which when executed by acomputer, cause the computer to execute operations, the operationscomprising: applying, as a condition that diagonal scan is applied to atransform block of variable block size, diagonal scan to all transformblocks of a set of variable block sizes, wherein 4×4 sub-blocks arescanned using a diagonal scan and the diagonal scan is applied insideeach 4×4 sub-block; and applying multi-level significance map decodingto the transform block.
 10. The non-transitory computer-readable mediumaccording to claim 9, further comprising: performing unified multi-levelsignificance map decoding on the transform block.
 11. The non-transitorycomputer-readable medium according to claim 10, wherein the set ofvariable block sizes comprises 4×4, 8×8, 16×16, and 32×32 blocks. 12.The non-transitory computer-readable medium according to claim 11,further comprising: forming the transform block based on recursivelysplitting a coding block into transform block by block partitioning.